View MT27508A0-FCCR-FV_6962732.PDF datasheet online --- IC-ON-LINE

Datasheet File OCR Text:

www.mellanox.com connectx ? -3 vpi single and dual qsfp+ port adapter card user manual p/n: mcx353a-fcbt, mcx353a-tcbt, mcx353a-q cbt, mcx354a-fcbt, mcx354a-tcbt, mcx354a-qcbt rev 1.6 www.datasheet.net/ datasheet pdf - http://www..co.kr/
mellanox technologies 350 oakmead parkway suite 100 sunnyvale, ca 94085 u.s.a. www.mellanox.com tel: (408) 970-3400 mellanox technologies, ltd. beit mellanox po box 586 yokneam 20692 israel www.mellanox.com tel: +972 (0)74 723 7200 ? copyright 2012. mellanox technol ogies. all rights reserved. mellanox?, mellanox logo?, bridgex? , connectx?, core-direct?, infinibridge?, infinihost?, infi niscale?, phyx?, switchx?, virtual protoc ol interconnect? and voltaire? ar e registered trademarks of me llanox technologies , ltd. connect- ib?, fabricit?, mlnx-os?, scalablehpc?, testx? , unbreakable-link?, ufm? and unified fabric manager? are trademarks of mellanox technologies, ltd. all other trademarks are propert y of their respective owners. document number: 3754 connectx?-3 vpi single and dual qsfp + port adapter card user manual rev 1.6 mellanox technologies 2 note: this hardware, software or te st suite product (?product(s)?) and its related documentation are provided by mellanox te chnologies ?as-is? with all faults of any kind and solely for the purpose of aiding the customer in testing applications that use the products in designated solutions. the customer's manufacturing test environmen t has not met the standards set by mellanox technologies to fully qualify th e product(s) and/or the system using it. therefore, mellanox technologies cannot and does not g uarantee or warrant that the pr oducts will operate with the highest quality. any express or implied warranti es, including, but not limited to, the implied warranties of merchantability, fitness for a particular purp ose and noninfringement are disclaimed. in no event shall mellanox be liable to customer or any third parties for any direct, indirect, special, exemplary, or consequential damages of any kind (including, but not limited to, payment for procurement of substitute goods or servic es; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in cont ract, strict liability, or tort (including negligence or otherwise) arising in any way from the use of the product(s) and related documentation even if advised of the possibility of such damage. www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 3 table of contents table of contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 list of tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 list of figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 revision history . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 about this manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 intended audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 related documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 document conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 technical support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 firmware and software updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 chapter 1 overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.1 supported network protocol standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.1.1 infiniband fdr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.1.2 infiniband fdr10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.1.3 infiniband qdr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.1.4 10 and 40 gigabit ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2 adapter cards covered in this manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.3 finding the guid/ mac and serial number on the adapter card . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.4 safety warnings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 chapter 2 adapter card interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.1 i/o interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.1.1 infiniband interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.2 ethernet interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.3 vpi port configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.4 pci express interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.5 led assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.1.6 i2c-compatible interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.2 power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.3 qsfp+ power level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.4 memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.4.1 system memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.4.2 spi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.4.3 eeprom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 chapter 3 adapter card installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.1 hardware requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.2 installation instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3 identify the card in your system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3.1 on windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3.2 on linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.4 cables and modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.4.1 cable installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.4.1.1 inserting a cable into the adapter card 26 3.4.1.2 removing a cable from the adapter card 27 chapter 4 driver software and firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.1 driver software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.1.1 linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.1.2 windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.2 port type management via driver sw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 4 4.2.1 linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.2.2 windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.3 flexboot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.4 nvidia gpudirect support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.4.1 hardware and software requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.4.2 installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.4.2.1 kernel installation 35 4.4.2.2 mlnx ofed drivers installation 35 4.4.2.3 nvidia driver installation 36 4.5 rdma over converged ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.5.1 hardware and software requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 4.6 updating adapter card firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 appendix a specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 a.1 mcx353a-qcbt specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 a.2 mcx354a-qcbt specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 a.3 mcx353a-tcbt specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 a.4 mcx354a-tcbt specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 a.5 mcx353a-fcbt specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 a.6 mcx354a-fcbt specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 a.7 board mechanical drawing and dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 a.8 regulatory statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 appendix b interface connectors pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 b.1 i2c-compatible connector pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 b.2 pci express x8 connector pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 b.3 qsfp+ connector pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 appendix c replacing a tall bracket with a short bracket . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 c.1 replacing a bracket. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 c.2 removing the existing bracket from the adapter card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 c.3 installing the new bracket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 appendix d avertissements de scurit d?installation (warnings in french) . . . . . . . . . . . . . . 54 appendix e sicherheitshinweise (warnings in german). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 appendix f advertencias de seguridad para la instalaci n (warnings in spanish) . . . . . . . . . 58 www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 6 list of figures figure 1: card product label (example) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 figure 2: mcx354a-[qft]cbt card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 figure 3: dual-port bracket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 figure 4: single-port bracket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 figure 5: physical and logical link indications (infiniband m ode) . . . . . . . . . . . . . . . . . . . . . . . 19 figure 6: i2c-compatible connector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 figure 7: pci device. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 figure 8: connector orientation (example) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 figure 9: support download assistant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 figure 10: mechanical drawing of the single-port mcx3 53a-[qft]cbt adapter card . . . . . . . 45 figure 11: mechanical drawing of the dual-port mcx354 a-[qft]cbt adapter card . . . . . . . . 46 figure 12: compatible connector plug and pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 figure 13: pcie connector pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 figure 14: connector and cage views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 figure 15: bracket screws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 figure 16: gasket location on adapter card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 figure 17: gasket installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 figure 18: placing the bracket on the card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 5 list of tables table 1: revision history table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 table 2: documents list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 table 3: single and dual-port qdr adapter cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 table 4: single and dual-port fdr10 adapter ca rds. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 table 5: single and dual-port fdr adapter cards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 table 6: supported port configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 table 7: unsupported port configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 table 8: physical and logical link indication (ethernet mode). . . . . . . . . . . . . . . . . . . . . . . . . . 19 table 9: jumper configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 table 10: mcx35[34]a-[qft]cbt vpd layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 table 11: mcx353a-qcbt specifications table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 table 12: mcx354a-qcbt specifications table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 table 13: mcx353a-tcbt specifications table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 table 14: mcx354a-tcbt specifications table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 table 15: mcx353a-fcbt specifications table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 table 16: mcx354a-fcbt specifications table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 table 17: i2c-compatible connector pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 table 18: connector pin number and name to signal name map. . . . . . . . . . . . . . . . . . . . . . . . . 50 www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 7 revision history this document was printed on november 14, 2012. table 1 - revision history table date rev comments/changes november 2012 1.6 ? updated web links in the following locations: october 2012 1.56 ?fixed table 9, ?jumper configuration,? on page 21 ? updated figures in section 4.1.2, ?windows,? on page 32 by pro- viding connectx-3 examples august 2012 1.4 ? added operational and non-opera tional temp erature and humid - ity level to the following tables: january 2012 1.2 ? minor edits ? updated led functions in section 2.1.5, ?led assignment,? on page 19 ? formatted specification tables in appendix a, ?specifications,? on page 39 october 2011 1.1 added new opns based on c onn ectx-3 step a1 devices july 2011 1.0 minor edits july 2011 0.10 first release - section 4.5.1, ?hardware an d software requirements,? on page 36 - table 11, ?mcx353a-qcbt sp ecifications table,? on page 39 - table 12, ?mcx354a-qcbt specifications table,? on page 40 - table 13, ?mcx353a-tcbt specif ications tabl e,? on page 41 - table 14, ?mcx354a-tcbt specif ications tabl e,? on page 42 - table 15, ?mcx353a-fcbt specif ications table,? on page 43 - table 16, ?mcx354a-fcbt specif ications table,? on page 44 - table 11, ?mcx353a-qcbt sp ecifications table,? on page 39 - table 12, ?mcx354a-qcbt sp ecifications table,? on page 40 - table 13, ?mcx353a-tcbt specif ications table,? on page 41 - table 14, ?mcx354a-tcbt specif ications table,? on page 42 - table 15, ?mcx353a-fcbt specif ications table,? on page 43 - table 16, ?mcx354a-fcbt specif ications table,? on page 44 www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 8 www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 8 about this manual this user manual describes mellanox technologies conn ectx?-3 vpi single and dual qsfp+ port pci express x8 adapter cards. it provides de tails as to the interfaces of the board, specifica - tions, required software and firmware for oper atin g the board, and relevant documentation. intended audience this manual is intended for the installer and user of these cards. the manual assumes basic familiarity with infiniband? and ethernet networks and architecture specifications. related documentation table 2 - mellanox firmware tools (mft) user manual document no. 2204ug user manual describing the set of mf t firmware manage ment tools for a single node. see http://www .mellanox.com => support => download firmware tools ibta specificati on release 1.2.1 infiniband architecture specification ieee std 802.3 specification this is the ieee ethernet specification http://standards.ieee.org/getieee802 pci express 3.0 specifications industry standard pci express 3.0 base and c ard el ectromechanical specifications documents list document conventions when discussing memory sizes, mb and mbytes are us ed in this document to mean size in mega bytes. the use of mb or mbits (small b) indicates size in mega bits. ib is us ed in this document to mean infiniband. in this document pcie is used to mean pci express. technical support customers who purchased mellanox products dir ectly from mellanox are invited to contact us through the following methods. ?url: http:/ /www.mellanox.com => support ? e-mail: su pport@mellanox.com ? tel: +1.408.916.0055 www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 9 customers who purchased mellanox m-1 global su pport services, please see your contract for details regarding technical support. customers who purchased mellanox products thro ugh a mellanox approved reseller should first seek assistance through their reseller. firmware and software updates the mellanox support downloader contains software, firmware and knowledge database informa - tion for mellanox products. access the data base from the mellanox support web page, http://www.mellanox.com => support or use the following link to go directly to the mellanox support down load assistant page, http://www.mellanox.com/ supportdownloader/. www.datasheet.net/ datasheet pdf - http://www..co.kr/
overview rev 1.6 mellanox technologies 10 1 overview this document is a user manual for mellanox technologies vp i adapter based on the con - nectx?-3 vpi integrated circuit device. the cards described in this manual have the following main features: ? virtual protocol interconnect (vpi) ? infiniband architecture specification v1.2.1 compliant ? ieee std. 802.3 compliant ? pci express 3.0 (1.1 and 2.0 compatible) through an x8 edge connector up to 8gt/s ? single and dual-port options available - esd protection on the network connector ? compliant with qsfp+ msa spec rev. 1.0 ? compatible with copper cables and optical cabl es with the use of qsfp connectors. support for sfp+ cables is available through qsa (quad to serial) ? core-direct? application off-load ? gpudirect application off-load ? rdma over converged ethernet (roce) ? end-to-end qos and congestion control ? hardware-based i/o virtualization ? tcp/udp/ip stateless off-load ? ethernet encapsulation (eoib) ? rohs-r6 compliant ? two bracket heights: short and tall 1.1 supported network protocol standards 1.1.1 infiniband fdr fdr is a pre-standard infiniband data rate, where each lane of a 4x port runs a bit rate of 14.0625gb/s with a 64b/66b encodi ng, resulting in an effective ba ndwidth of 54.54gb/s. the fdr physical layer is an ibta specified physical la yer using different block types, deskew mechanism and framing rules. 1.1.2 infiniband fdr10 fdr10 is an infiniband transport and link laye r running over a physical layer using the pcs encoding of 64b/66b and a bit rate of 10.3125gb/s per lane. consequently, the effective bandwidth is 25% higher than the infiniband-specification standard qdr rate, which uses an 8b/10b encod - ing. the fdr10 implementation on the adapter device can use the optional forward error correc - tion (fec) supported by the fdr pre-standard. www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 11 1.1.3 infiniband qdr qdr is a standard infiniband data rate, where each lane of a 4x port runs a bit rate of 10gb/s. 1.1.4 10 and 40 gigabit ethernet 10 and 40 gigabit ethernet are ethernet networ k protocol standards for lans specifying data transfer rates of 10 and 40gb/s, respectively. mellanox adapters comply with the following ieee 802.3* standards: ? ieee std 802.3-2008 ethernet ? ieee std 802.3ae 10 gigabit ethernet ? ieee std 802.3ba 40 gigabit ethernet ? ieee std 802.3ad link aggregation and failover www.datasheet.net/ datasheet pdf - http://www..co.kr/
overview rev 1.6 mellanox technologies 12 1.2 adapter cards cover ed in this manual table 3 lists the single and du al-port qdr adapter cards. table 4 lists the single and dual-port fdr10 adapter cards table 5 lists the single and dual-port fdr adapter cards. table 3 - single and dual-port qdr adapter cards ordering part number (opn) pci express serdes speed data transmis- sion rate/ # of ports rohs adapter ic part number device id (decimal) mcx353a-qcbt pcie 3.0 x8 8gt/s 40 gb/s qdr ib, 10gige single- port qsfp+ r6 mt27508a1-fccr-qv mcx354a-qcat (legacy product) pcie 3.0 x8 8gt/s 40 gb/s qdr ib, 10 gige dual-port qsfp+ r6 mt27508a0-fccr-qv mcx354a-qcbt pcie 3.0 x8 8gt/s 40 gb/s qdr ib, 10gige dual-port q sfp+ r6 mt27508a1-fccr-qv table 4 - single and dual-port fdr10 adapter cards ordering part number (opn) pci express serdes speed data transmis- sion rate/ # of ports rohs adapter ic part number device id (decimal) mcx353a-tcbt pcie 3.0 x8 8gt/s 40 gb/s fdr10 ib, 10gige sin - gle-port qsfp+ r6 mt27508a1-fccr-tv mcx354a-tcat (legacy product) pcie 3.0 x8 8gt/s 40 gb/s fdr10 ib, 10gige dual- port qsfp+ r6 mt27508a0-fccr-tv mcx354a-tcbt pcie 3.0 x8 8gt/s 40 gb/s fdr10 ib, 10gige dual- port qsfp+ r6 mt27508a1-fccr-tv ? 4099 for physical function ? 4100 for virtual function ? 4099 for physical function ? 4100 for virtual function www.datasheet.net/ datasheet pdf - http://www..co.kr/
table 5 - single and dual-port fdr adapter cards ordering part number (opn) pci express serdes speed data transmis- sion rate/ # of ports rohs adapter ic part number device id (decimal) mcx353a-fcbt pcie 3.0 x8 8gt/s 56 gb/s fdr ib, 40gige single- port qsfp+ r6 mt27508a1-fccr-fv mcx354a-fcat (legacy product) pcie 3.0 x8 8gt/s 56 gb/s fdr ib, 40gige dual-port q sfp+ r6 MT27508A0-FCCR-FV mcx354a-fcbt pcie 3.0 x8 8gt/s 56 gb/s fdr ib, 40gige dual-port q sfp+ r6 mt27508a1-fccr-fv connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 13 1.3 finding the guid/ mac and seri al number on the adapter card each mellanox adapter card has a label on the print side that shows the card serial number, the card mac for the ethernet protocol and the card gui d for the infiniband protocol. vpi cards have both a guid and a mac (derived from the guid). figure 1: card product label ( example) port 1 uses the guid or mac described on the label. to obtain the guid or mac for port 2, add 1 to that of port 1. 1.4 safety warnings below is a list of safety warnings in english. for safety warnings in othe r languages, please refer to the appendices in this user manual. ? 4099 for physical function ? 4100 for virtual function www.datasheet.net/ datasheet pdf - http://www..co.kr/
overview rev 1.6 mellanox technologies 14 1. installation instructions read all installation instructions before c onnecting the equipmen t to the power source. 2. over-temperature this equipment should not be operated in an area with an ambient temperature exceed - ing the maximum recommended: 55c (131f). to guarantee proper air flow, allow at least 8cm (3 inches) of clearance around the ven - tilation openings. 3. during lightning - electrical hazard during periods of lightning activity, do not work on the equipment or connect or dis - connect cables. 4. copper cable connecting/disconnecting some copper cables are heavy and not flexible, as such they should be carefully attached to or detached fro m the connectors. refer to th e cable manufacturer for spe - cial warnings and instructions. 5. equipment installation this equipment should be inst alled, replaced, or serviced on ly by trained and qualified personnel. 6. disposal of this equipment should be in accordance to all national laws and regula - tions. equipment disposal 7. this equipment should be installed in compliance with local and national electrical codes. local and national electrical codes www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 15 8. hazardous radiation exposure caution ? use of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure. class 1 laser product and reference to the most recent laser standards: iec 60 825-1:1993 + a1:1997 + a2:2001 and en 60825-1:1994+a1:1996+ a2 :2001 www.datasheet.net/ datasheet pdf - http://www..co.kr/
adapter card interfaces rev 1.6 mellanox technologies 16 2 adapter card interfaces 2.1 i/o interfaces each adapter card includes the following interfaces: ? qsfp+ port(s) ? pci express (pcie) x8 edge connector ? i/o panel leds ?i 2 c-compatible conn ector (for debug) figure 2: mcx354a-[qft]cbt card the adapter cards include special circuits to pr otect from esd shocks to the card/server when plugging copper cables. for dual-port cards, port 1 connects to connector 1 of the device, while port 2 connects to connec - tor 2 of the device. see figure 3 for the mechanical drawing of a dual-port bracket and figure 4 for the mechanical drawing of a single -port bracket. www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 17 figure 3: dual-port bracket figure 4: single-port bracket www.datasheet.net/ datasheet pdf - http://www..co.kr/
adapter card interfaces rev 1.6 mellanox technologies 18 2.1.1 infiniband interface the network ports of the connectx?-3 adapter cards are compliant with the infiniband architecture specification, release 1.2.1. infiniband traffic is transmitted through the cards' qsfp+ connectors. 2.1.2 ethernet interface the network ports of the connec tx?-3 adapter cards are compliant with the ieee 802.3 ethernet standards listed in section 1.1.4, ?10 and 40 gigabit ethernet,? on page 11 . ethernet traffic is transmitted through the cards' qsfp+ connectors. for connecting to an sf p+ interface, you can use mellanox qsa (qsfp to sfp+) adapte r module (to be ordered separately). 2.1.3 vpi port configuration vpi ports are auto-sensing but can be manually configured using a script. table 6 lists the configuratio n s supported by dual-port vpi adapter cards. table 6 - supported port configurations port 1 port 2 ethernet ethernet ib ib auto-sensing auto-sensing ib ethernet ib auto-sensing auto-sensing ethernet table 7 lists the configurations not suppor ted by dual-port vpi adapter cards. table 7 - unsupported port configurations port 1 port 2 ethernet ib ethernet auto-sensing auto-sensing ib for port type configuration instructions please refer to section 4.2, ?port type management via driver sw,? on page 32 . 2.1.4 pci express interface the connectx?-3 adapter cards support pci expr ess 3.0 (1.1 and 2.0 compatible) throughanx8 edge connector. the devi ce can be either a master initiating the pci express bus operations or a slave responding to pci bus operations. www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 19 2.1.5 led assignment there are two i/o leds per port in dual-port designs. see figure 5 for different led functions in infiniband mode and table 8 for different led functio ns in ethernet mode. figure 5: physical and logical link indications led function green - physical link ? constant on indicates a good phys- ical link ? blinking indicates a problem with the physical link ? if neither led is lit, then the phys- ical link has not been established yellow - logical (data activity) link ? a constant yellow indicates a valid logical (data activ ity) link without data transfer. ? a blinking yellow indicates a valid logical link with data transfer ? if only the green led is lit and the yellow led is off, then the logical link has not been established port 2 port 1 (infiniband mode) table 8 - physical and l ogical link indication led function green - physical link ? constant on indicate s a good physical link ? if neither led is lit, then the phys ical link has not been established yellow - logical (data activity link ) ? a blinking yellow indicate s activity (data transfer) ? stays off when there is no activity (ethernet mode) www.datasheet.net/ datasheet pdf - http://www..co.kr/
the short bracket has the same port a nd led footprints as the tall bracket. adapter card interfaces rev 1.6 mellanox technologies 20 2.1.6 i 2 c-compatible interface a three-pin header on the adapte r cards is provided as the i 2 c-compatible interface. see figure 11, ?mechanical drawing of the dual-port mcx354a-[qft]cbt adapter card,? on page 46 for the location on the board. figure 6: i 2 c-compatible connector 2.2 power all adapter cards receive 12v and 3.3v power from the pci express edge connector. all other required power voltages are generated by on-board switch mode regulators. see ?specifications? on page 39. 2.3 qsfp+ power level the adapter cards support power levels according to sff committ ee sff-84366 specification for qsfp+ (quad small form-factor pluggable) transceiver. 2.4 memory the adapter cards support multiple memory devices through the pcie, spi (flash) and i 2 c inter - faces. 2.4.1 system memory the adapter cards ut ilize the pci express interface to store an d access infiniband and/or ethernet fabric connection information and pa cket data on the system memory. 2.4.2 spi each of the adapter cards includes one 16mb spi flash device (m25px16-vmn6p device by st microelectronics) accessible via the spi interface of the connectx?-3 vpi device. www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 21 when a jumper is inserted into the drill holes in the adapter it indicates to the device that the on- board flash device should not be used for boot instructions. contact your mellanox support repre - sentative should you need to use this jumper . table 9 provides information on this jumper. the jumper location on the board is illustrated in appendix a.7, ?board mechanical drawing and dimensions,? on page 45 . table 9 - jumper configuration description option card default configuration flash present/ not present connection open ? flash present connection shorted ? flash not present connection open ? flash present 2.4.3 eeprom each board incorporates an eeprom that is accessible through the i 2 c-compatible interface. the eeprom capacity is 4kb. the eeprom is used for storing the vital product data (vpd). the pci vpd (vital product data) la yout for ea ch of the adapter cards complies with the format defined in the pci 3.0 specification, appendix i. table 10 - mcx35[34]a-[qft]cbt vpd layout offset (decimal) item value format description 0 large resource type id s tring tag (0x02) 0x82 1 length [7:0] lsb 0x18 2 length [15:8] msb 0x0 3 data cx354a - con - nectx-3 qsfp str 27 large resource type vpd-r t ag (0x10) 0x90 28 length [7:0] lsb 0x4f 29 length [15:8] msb 0x00 30 vpd keyword pn str add in card part number example: mcx354a-fcbt 32 length 0x15 33 part number pn %str_spc 54 vpd keyword ec str engineering change level of the card (rev) www.datasheet.net/ datasheet pdf - http://www..co.kr/
adapter card interfaces rev 1.6 mellanox technologies 22 56 length 0x2 57 revision rv %str pcb revision example: ?a1? 59 vpd keyword sn str serial number 61 length 0x18 62 serialnumber sn %str_spc ?00..00xxxx..xx? 86 vpd keyword v0 str misc information 88 length 0x10 89 data pcie gen3 x8 str_spc 105 vpd keyword rv str 107 length 0x1 108 data 0,107 %cs0 109 large resource type vpd-w tag (0x11) 0x91 110 length [7:0] lsb 0x8f 111 length [15:8] msb 0xf 112 vpd keyword v1 str efi driver version 114 length 0x6 115 data n/a str_spc 121 vpd keyword ya str asset tag 123 length 0x18 124 data n/a str_spc ?n/a? 148 vpd keyword rw str remaining read/write area 150 length 0x69 151 data str_zero reserved (0x00) 256 vpd keyword rw str remaining read/write area 258 length 0xfd 259 data str_zero reserved (0x00) 512 vpd keyword rw str remaining read/write area 514 length 0xfd 515 data str_zero reserved (0x00) 768 vpd keyword rw str remaining read/write area 770 length 0xfd 771 data str_zero reserved (0x00) 1024 vpd keyword rw str remaining read/write area 1026 length 0xfd 1027 data str_zero reserved (0x00) 1280 vpd keyword rw str remaining read/write area 1282 length 0xfd 1283 data str_zero reserved (0x00) 1536 vpd keyword rw str remaining read/write area 1538 length 0xfd 1539 data str_zero reserved (0x00) table 10 - mcx35[34]a-[qft]cbt vpd layout offset (decimal) item value format description www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 23 1792 vpd keyword rw str remaining read/write area 1794 length 0xfd 1795 data str_zero reserved (0x00) 2048 vpd keyword rw str remaining read/write area 2050 length 0xfd 2051 data str_zero reserved (0x00) 2304 vpd keyword rw str remaining read/write area 2306 length 0xfd 2307 data str_zero reserved (0x00) 2560 vpd keyword rw str remaining read/write area 2562 length 0xfd 2563 data str_zero reserved (0x00) 2816 vpd keyword rw str remaining read/write area 2818 length 0xfd 2819 data str_zero reserved (0x00) 3072 vpd keyword rw str remaining read/write area 3074 length 0xfd 3075 data str_zero reserved (0x00) 3328 vpd keyword rw str remaining read/write area 3330 length 0xfd 3331 data str_zero reserved (0x00) 3584 vpd keyword rw str remaining read/write area 3586 length 0xfd 3587 data str_zero reserved (0x00) 3840 vpd keyword rw str remaining read/write area 3842 length 0xfc 3843 data str_zero reserved (0x00) 4095 small resource type end tag (0x11) 0x78 table 10 - mcx35[34]a-[qft]cbt vpd layout offset (decimal) item value format description www.datasheet.net/ datasheet pdf - http://www..co.kr/
adapter card installation rev 1.6 mellanox technologies 24 3 adapter card installation 3.1 hardware requirements a system with a pci express x8 slot is required for in stalling the card. 3.2 installation instructions read all installation instruc tions before connecting the equipment to the power source. to change a tall bracket to a short bracket see replacing a tall bracket with a short bracket on page 51 . the adapter cards listed above are standard pci express cards, each with a standard x8 edge con - nector. the cards require a pci express x8. please consult the host ma chine documentation for instructions on how to install a pci express card. if the card is installed in a pc i slot with less lanes than th e card requires then the adapter card will not provide the optimum data transfer. 3.3 identify the card in your system 3.3.1 on windows 1. open device manager on the server. click start => run, and then enter ?devmgmt.msc?. 2. expand system devices and locate yo ur mellanox connectx-3 adapter card. 3. select properties to display th e adapter card prope rties window. 4. click the details tab and select d evice instance id (windows 2003) or hardware ids (w indows 2008/r2) from the property pull-down menu. www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 25 figure 7: pci device 5. in the value display box, check the fields ven and dev (fields are separated by ?&?). in the display example above, notice the sub-string ?pci\ven_15b3&dev_1003?: ven is equal to 0x15b3 ? this is the vendor id of mellan ox technologies; and dev is equal to 1003 ? this is a valid mellanox te chnologies pci device id. if the pci device does not have a mellanox adapter id, return to step 4 to check another device. the list of mellanox technologies pci de vice ids can be found in the pci id reposi - tory at http://pci-ids.ucw.cz/read/pc/15b3. 3.3.2 on linux get the device location on the pc i bus by running lspci and loca ting lines with the string ?mella- nox technologies?: > lspci |grep -i mellanox 27:00.0 network controller: mellanox technologies mt27500 fam- ily [connectx-3] www.datasheet.net/ datasheet pdf - http://www..co.kr/
adapter card installation rev 1.6 mellanox technologies 26 3.4 cables and modules please refer to ?mellanox products approved cable lists? at www.mellanox.com/related-docs/ user_manuals/mellanox_approved_cables.pdf. 3.4.1 cable installation 1. all cables can be inserted or removed with th e unit powered on. to insert a cable, press the connector into the port receptacle until the conn ector is firmly seated. 2. after inserting a cable into a port, the green led indicator wil l light when the physical connection is established (that is, when the unit is powered on an d a cable is plugged into the port with the other end of the connector plugged into a functioning port). 3. after plugging in a cable, lock the connec tor using the latching mechanism particular to the cable vendor. when a logical connection is made the yellow led will come on. when data is being transferred the yellow led will blink. when installing cables make sure that the latches engage. always install and remove cables by pushing or pulling the cable and connector in a straight line with the card. 4. care should be taken as not to impede the air exhaust flow through the ventilation holes. cable lengths which allow for routing horizontal ly around to the side of the chassis before bending upward or downward in the rack should be used 5. to remove a cable, disengage the locks and sl owly pull the connector away from the port receptacle. both led indicators will turn off when the cable is unseated. cables, especially long copper cables, can weigh a substantial amount. make sure that the weight of the cable is supported on its own and is not hanging from the adapter card. 3.4.1.1 inserting a cable into the adapter card 1. support the weight of the cable before connec ting the cable to the adap ter card. do this by using a cable holder or tying the cable to the rack. 2. determine the correct orientation of the connec tor to the card before inserting the connector. do not try and insert the connector upside down. this may damage the adapter card. 3. insert the connector into the adapter card. be careful to insert the conn ector straight into the cage. do not apply any torque, up or down, to the connector cage in the adapter card. www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 27 4. make sure that the connector locks in place. figure 8: connector orientation (example) 3.4.1.2 removing a cable from the adapter card 1. pull on the latch release mechanism thereby unl atc hing the connector an d pull the connector out of the cage. 2. do not apply torque to the connector when removing it from the adapter card. 3. remove any cable supports that were used to support the cable?s weight. www.datasheet.net/ datasheet pdf - http://www..co.kr/
driver software and firmware rev 1.6 mellanox technologies 28 4 driver software and firmware 4.1 driver software 4.1.1 linux for linux, download and install the latest open fabrics enterprise distribution (ofed) software package available via the mellanox web site at: http://www.mellanox.com => products => soft - ware/drivers => infiniband & vp i software/drivers => mellanox ofed => download. follow the installation instructions incl uded in the download package (a lso available from the download page).to ensure that commun ication has been established follow the instructions below. check the link status first check the network interface name by running the ?ifc onfig ?a? command > ifconfig ?a eth0 link encap:ethernet hwaddr 5c:f3:fc:5f:40:ea inet addr:10.7.5.170 bcast:10.7.7.255 mask:255.255.252.0 inet6 addr: fe80::5ef3:fcff:fe5f:40ea/64 scope:link up broadcast running multicast mtu:1500 metric:1 rx packets:20009 errors:0 dropped:0 overruns:0 frame:0 tx packets:4492 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 rx bytes:7005902 (6.6 mib) tx bytes:399581 (390.2 kib) > host# ethtool eth0 settings for eth0: supported ports: [ tp ] supported link modes: 1000baset/full 10000baset/full supports auto-negotiation: yes advertised link modes: 1000baset/full 10000baset/full run: ethtool www.datasheet.net/ datasheet pdf - http://www..co.kr/
advertised auto-negotiation: n speed: 1000mb/s duplex: full port: twisted pair phyad: 0 transceiver: internal auto-negotiation: on supports wake-on: g wake-on: d link detected: yes connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 29 to check the ib link status, for ib and vpi cards, run ? ibstat ? and focus on the physical state attributes. example: > host# ibstat lca 'mlx4_0' ca type: mt4099 number of ports: 2 firmware version: 2.10.2000 hardware version: 0 node guid: 0x0002c903002fefe0 system image guid: 0x0002c903002fefe3 port 1: state: active physical state: linkup rate: 56 base lid: 4 lmc: 0 sm lid: 12 capability mask: 0x02514868 port guid: 0x0002c903002fefe1 link layer: infiniband port 2: state: active www.datasheet.net/ datasheet pdf - http://www..co.kr/
physical state: linkup rate: 56 base lid: 8 lmc: 0 sm lid: 12 capability mask: 0x02514868 port guid: 0x0002c903002fefe2 link layer: infiniband driver software and firmware rev 1.6 mellanox technologies 30 check the ofed version to get the version of the running mellanox ofed/bxof ed, run th e following command: troubleshooting mlnx_ofed installation for troubleshooting driver installation, please c heck mellanox ofed driver user manual at: http://www.mellanox.com => support => infiniband software and drivers => mellanox ofed. loading the ethernet driver by default, the mellanox ofed stack loads mlx4_e n. run ?lsmod? to verify that the module is listed. example: if you don?t see the mlx4_en driver, run: ? modprobe mlx4_en? another option is to use the comman d below to see which modules are active. for example: # ofed_info |head -1 mlnx_ofed_linux-1.5.3-0.1.6 (ofed-1.5.3-0.1.6): host# lsmod |grep mlx4_en mlx4_en 114184 0 mlx4_core 156512 2 mlx4_ib,mlx4_en edit ?/etc/infiniband/openib.conf? which modules needs to load from a service. # load mlx4_en module www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 31 the ?usage: openibd {start|stop|restart|status}? co mmand to modify this f ile and thereby control the drivers. ethernet driver usage and configuration to assign an ip address to the interface run: where 'n' is the os assigned interface number. ? to check driver and device information run: example: > ethtool -i eth0 driver:mlx4_en (mt_1020110019_cx-3) version: 1.5.6.33 (oct 2011) firmware-version:2.10.0000 bus-info: 0000:07:00.0 ? the mlx4_en parameters can be found under /sys/module/mlx4_en (or /sys/module/mlx4_en/ pa rameters, depending on the os) and can be listed using the command: to set non-default values to module parameters, the following line should be added to the file/etc/ modprobe.conf: ethernet network tuning to improve network performance by tuning your network see the mellanox performance tuning guide located at: http://www.mellanox.com => products => software/drivers => ethernet sw/drivers => linux driver mlx4_en_load=yes #> ifconfig eth #> ethtool -i eth #> modinfo mlx4_en "options mlx4_en = = ..." www.datasheet.net/ datasheet pdf - http://www..co.kr/
driver software and firmware rev 1.6 mellanox technologies 32 4.1.2 windows for windows, download and inst all the latest mellanox winof vpi for windows software pack - age available via the mellanox web site at: http://www.mellanox.com => products => software/ drivers =>infiniband & vpi software/drivers => mellanox winof vpi => download. follow the installation instructions incl uded in the download package (a lso available from the download page). to ensure that communication has been established follow the instructions below. displaying the device manager will show the me llanox adapter devices an d an ipoib (network) device for each port. note that mlnx_vpi configur es the ports of vpi cards ? at installation time ? to run the infiniband protocol. 4.2 port type management via driver sw 4.2.1 linux if you wish to change the port type, use the connec tx_port_config script afte r the driver is loaded. the script is installed as part of the mellanox of ed for linux package (under /sbin). see the mel- lanox ofed for the linux user manual available at http://www.mellanox.com => support => infiniband products => mellanox ofed. running "/ sbin/connectx_por t_config -s" will show the current port configuration for all connectx?-3 devices. port configuration is saved in the file: /etc/infi niband/connectx.conf. this saved configuration is restored at driver restart only if do ne via "/etc/init.d/openibd restart". possible port types are: ? "eth" - always ethernet ? "ib" - always infiniband ? "auto" - link sensing mode - detect the port ty pe based on the attached network type. if no link is detected, the driver retrie s link sensing every few seconds. the port link type can be configured for each devi ce in the system at ru n time using the "/sbin/ connectx_port_config" script. this ut ility will prompt for the pci device to be modified (if there is only one it will be selected automatically). at th e next stage the user will be prompted for the desired mode for each po rt. the desired port configuration will then be set for the selected device. note: this utility also h as a non-interactive mode: /sbin/connectx_port_config [[-d|--device ] -c|--conf ] 4.2.2 windows after installing mellanox winof vp i for windows on your machine, you can change a port's pro- tocol configuration. the following steps de scribe how to configure the port type: step 1 display the device manager and expand ?system devices? www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 33 step 2. right-click on the mellanox connectx vpi netw ork adapter and left-clic k properties. select the port protocol tab from the properties sheet. note: the "port protocol? tab is displayed only if the nic is a vpi (ib and eth). if the nic is either only ib or eth, the tab will not be shown. note: the figure below is an ex ample of the displayed port prot ocol sheet for a dual-port vpi adapter card. www.datasheet.net/ datasheet pdf - http://www..co.kr/
driver software and firmware rev 1.6 mellanox technologies 34 step 3. in this step, you can perform two different func tions: (a) choose the desired port protocol for the available port(s), and (b) activate or d eactivate the wsd, nd , and/or sdp ulps note: ib must be always the first port in port 1. if you choose eth as your first port in port 1, then the second port in port 2 can be only eth. note: wsd is not supported in windows 7. con sequently , on this os the wsd checkbox is grayed out and cannot be selected. 4.3 flexboot flexboot enables remote boot over ethernet, boot over ethernet (boe), boot over infiniband (boib) or boot over iscsi (bo-iscsi). this t echnology is based on th e preboot execution envi - ronment (pxe) standard specification, and flexbo ot so ftware is based on the open source ipxe project (see www.ipxe.org). for more information go to => products => so ftware/drivers => infiniband & vpi software/dri vers => flexboot => download. 4.4 nvidia gpudirect support utilizing the high computational power of the graphics processing unit (gpu), the gpu-to-gpu method has proven valuable in various areas of science and technology. the mellanox con - nectx?-3 based adapter cards provide the required high throughput and low latency for gpu-to- gpu communications. www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 35 4.4.1 hardware and software requirements software: ? operating system: rhel5.4 2.6.18-164.el5 x86_64 or later ? mellanox ofed with gpudirect support ? nvidia development driver for li nux version 195.36.15 or later hardware: ? mellanox connectx?-3 adapter card ? nvidia tesla series 4.4.2 installation for complete installatio n instructions visit: www.mellanox.com => products => software/drivers => infiniband & vpi sw/driver => mella nox ofed gpudirect => download. 4.4.2.1 kernel installation use red hat package manager (rpm) to install the kernel rpm files: 4.4.2.2 mlnx ofed drivers installation the mlnx driver is called mlnx_ofed_linux-nvidia-1.5.1. - install the required rpms, for example, run: # rpm --force -ivh *.rpm - modify the boot loader configuration file if needed (e.g., edit /etc/grub.conf) - reboot the machine with the new kernel # reboot - mount the iso file: # mount -o ro, loop mlnx_ofed_.iso /mnt - run the installation script: # /mnt/mlnxofed install - restart the driver: # /etc/init.d/openibd restart - to make sure that gpudirect is enabled, run: # cat /sys/module/ib_core/parameters/gpu_direct_enable => 1 www.datasheet.net/ datasheet pdf - http://www..co.kr/
driver software and firmware rev 1.6 mellanox technologies 36 the number of shared pages by gpudirect is reported under: 4.4.2.3 nvidia driver installation install nvidia development driver for linux x86_64, available under: for example: to make sure that the nvidia driver was installed successfully: 4.5 rdma over converged ethernet connectx?-3 connected to an ethe rnet fabric provides all of the basic nic functionality plus rdma over converged ethernet (roce). roce u tilizes advances in data center bridging (dcb) to enable efficient and low cost implementations of rdma over ethernet, supporting the entire breadth of rdma and low latency features. this includes reliable connected service, datagram service, rdma and send/receive seman tics, atomic operations, user level multicast, user level i/o access, kernel bypass, and zero copy . this capability is enabled wh en using the mellanox ofed or winof vpi drivers. connectx?-3 en with roce based network manageme nt is the same as that for any ethernet and dcb-based network management, eliminating the need for it managers to learn new technolo - gies. 4.5.1 hardware and software requirements software: ? mellanox ofed 1.5.1 or winof 2.1.2 or later. to download these packages go to http:// www.mellanox.com => products => software/drivers => infiniband & vpi sw/driver. # cat /sys/module/ib_core/parameters/gpu_direct_shares http://developer.nvidia.com/object/cuda_3_0_downloads.html - run: devdriver_3.0_linux_64_195.36.15.run - follow the installation wizard instructions - load nvidia driver: # modprobe nvidia - check the driver version, for example: # cat /proc/driver/nvidia/version => version 195.36.15 (or later) www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 37 hardware: ? connectx?-3 card 4.6 updating adapter card firmware each card is shipped with the latest version of qualified firmware at th e time of manufacturing. firmware is updated occasionally, and the mo st recent firmware can be obtained from: => support => download firmware. check that the firmware on your card is the latest found on the mellanox site, if not update to the late st version found on the mellanox web site. firmware can be updated on the stand alone single card using the flint tool of the mellanox firm - ware tools (mft) package. this package is available fo r download, along with its user manual, from the mellanox firmware tools page. see http://www.mellanox.com => support => download firmware tools. a firmware binaries table lists a binary file per ad apter card. the file name of each such binary is composed by combining the firmware name, the fi rmware release version, and the card part num - ber. please contact mellanox system support if you cannot find the fi rmware binary for your adapter card. the following steps describe how to retrieve th e psid (firmware identific ation) and programmed firmware version of your adapter card. they also describe how to update the card with the latest firmware version available. 1. retrieve the psid and firmware version: a. install the mft package. the package is available at http://www.mellanox.com => products => software/drivers => infiniband & vpi softwa re/drivers => firmware tools. make sure to download the package corresponding to your computer?s operating system. b. enter: mst start. c. get the mellanox mst device name using the command "mst status". the mst device name will be of the form: /dev/mst/mt4099_pci_cr0. d. get the psid (firmware identif ication) and programmed firmwa re version using the command. 2. compare the programmed firmware ve rsion with the latest available. > flint -d /dev/mst/mt4099_pci_cr0 q image type: connectx fw version: 2.9.4000 device id: 4099 chip revision: 0 description: node port1 port2 sys image guids: 000002c900000200 000002c900000201 000002c900000202 000002c900000203 macs: 000002c90200 000002c90201 board id: (mt_1020110019) vsd: psid: mt_1020110019 www.datasheet.net/ datasheet pdf - http://www..co.kr/
driver software and firmware rev 1.6 mellanox technologies 38 a. go to mellanox?s web site: http://www.mellanox.co m/supportdownloader. see figure 9. b. enter your card psid to display the latest firmware. figure 9: support download assistant 3. if a newer firmware version ex ists for your adapter card, up date the firmware as follows: a. download the firmware image zip file fro m the download center (see step 2a above). b. unzip the firmware image. c. burn the firmware image. enter: d. reboot the computer. e. enter: mst start. f. verify that the card firmware was updated successfully. > flint -d /dev/mst/mt4099_pci_cr0 -i burn > flint -d /dev/mst/mt4099_pci_cr0 q image type: connectx fw version: 2.9.4100 device id: 4099 ... www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 39 appendix a: specifications a.1 mcx353a-qcbt specifications table 11 - mcx353a-qcbt specifications table physical size: 2.12in. x5.6 in. (53.59mm x 142.25 mm) connector: qsfp+ 40gb/s infiniband (copper and optical) protocol support infiniband: ibta v1.2.1 auto-negotiation a a. the connectx-3 adapters supplement the ibta auto-n egotiation specification to get better bit error rates and longer cable reaches. this s upplemental feature only initiates when connected to another mellanox infiniband product. : 1x/2x/4x sdr (2.5gb/s per lane ), ddr (5gb/s per lane), qdr (10gb/s per lane) ethernet: 10gbase-cx4, 10gbase-r, and 1000base-r data rate: 40gb/s qdr? infiniband 1/10gb/s ? ethernet pci express gen3: serdes @ 8.0gt/s, 8 lanes (2.0 and 1.1 compatible) power and environmental voltage: 12v, 3.3v typ power: passive cables 5.63w active cables 6.28w b b. these power numbers were measured using mellanox cable mc22062xx-0xx-x. max power: passive cables 6.61w active cables 7.36w temperature: operational 0c to 55c non-operational 0c to 70c humidity: 90% relative humidity c c. for both operational a nd non-operational states air flow: 200lfm d d. air flow is measured ~1? from the heat sink between the heat sink and the cooling air inlet. regulatory emc: refer to the following link: www.mellanox.com/related-docs/user_manuals/regulator y_and_compliance_guide.pdf safety: iec/en 60950-1:2006 etsi en 300 019-2-2 iec 60068-2- 64, 29, 32 rohs: rohs-r6 cable support pplease refer to "mellanox produc ts approved cabl e lists" at: www.mellanox.com/related -docs/user_manuals/m ellanox_approved_cables.pdf www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 40 a.2 mcx354a-qcbt specifications table 12 - mcx354a-qcbt specifications table physical size: 2.71n. x5.6 in. (68.90mm x 142.25 mm) connector: qsfp+ 40gb/s infiniband (copper and optical) protocol support infiniband: ibta v1.2.1 auto-negotiation a a. the connectx-3 adapters supplement the ibta auto-n egotiation specification to get better bit error rates and longer cable reaches. this s upplemental feature only initiates when connected to another mellanox infiniband product. : 1x/2x/4x sdr (2.5gb/s per lane ), ddr (5gb/s per lane), qdr (10gb/s per lane) ethernet: 10gbase-cx4, 10gbase-r, and 1000base-r data rate: 40gb/s qdr? infiniband 1/10gb/s ? ethernet pci express gen3: serdes @ 8.0gt/s, 8 lanes (2.0 and 1.1 compatible) power and environmental voltage: 12v, 3.3v typ power: passive cables 6.78w active cables 8.08w b b. these power numbers were measured using mellanox cable mc22062xx-0xx-x. max power: passive cables 7.76w active cables 9.26w temperature: operational 0c to 55c non-operational 0c to 70c humidity: 90% relative humidity c c. for both operationsal a nd non-operational states air flow: 200lfm d d. air flow is measured ~1? from the heat sink between the heat sink and the cooling air inlet. regulatory emc: refer to the following link: www.mellanox.com/related-docs/user_manuals/regulator y_and_compliance_guide.pdf safety: iec/en 60950-1:2006 etsi en 300 019-2-2 iec 60068-2- 64, 29, 32 rohs: rohs-r6 cable support please refer to "mellanox produc ts approved cable lists" at: www.mellanox.com/related -docs/user_manuals/m ellanox_approved_cables.pdf www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 41 a.3 mcx353a-tcbt specifications table 13 - mcx353a-tcbt specifications table physical size: 2.12in. x5.6 in. (53.59mm x 142.25 mm) connector: qsfp+ 40gb/s infiniband (copper and optical) protocol support infiniband: ibta v1.2.1 auto-negotiation a a. the connectx-3 adapters supplement the ibta auto-n egotiation specification to get better bit error rates and longer cable reaches. this s upplemental feature only initiates when connected to another mellanox infiniband product. : 1x/2x/4x sdr (2.5gb/s per lane ), ddr (5gb/s per lane), qdr (10gb/s per lane), fdr 10 (1 0.3125gb/s per lane) ethernet: 10gbase-cx4, 10gbase-r, and 1000base-r data rate: 40gb/s fdr10 ? infiniband 1/10gb/s ? ethernet pci express gen3: serdes @ 8.0gt/s, 8 lanes (2.0 and 1.1 compatible) power and environmental voltage: 12v, 3.3v typ power: passive cables 5.69w active cables 6.34w b b. these power numbers were measured using mellanox cable mc22062xx-0xx-x. max power: passive cables 6.67w active cables 7.42w temperature: operational 0c to 55c non-operational 0c to 70c humidity: 90% relative humidity c c. for both operational a nd non-operational states air flow: 200lfm d d. air flow is measured ~1? from the heat sink between the heat sink and the cooling air inlet. regulatory emc: refer to the following link: www.mellanox.com/related-docs/user_manuals/regulator y_and_compliance_guide.pdf safety: iec/en 60950-1:2006 etsi en 300 019-2-2 iec 60068-2- 64, 29, 32 rohs: rohs-r6 cable support please refer to "mellanox produc ts approved cable lists" at: www.mellanox.com/related -docs/user_manuals/m ellanox_approved_cables.pdf www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 42 a.4 mcx354a-tcbt specifications table 14 - mcx354a-tcbt specifications table physical size: 2.71n. x5.6 in. (68.90mm x 142.25 mm) connector: qsfp+ 40gb/s infiniband (copper and optical) protocol support infiniband: ibta v1.2.1 auto-negotiation a a. the connectx-3 adapters supplement the ibta auto-n egotiation specification to get better bit error rates and longer cable reaches. this s upplemental feature only initiates when connected to another mellanox infiniband product. : 1x/2x/4x sdr (2.5gb/s per lane ), ddr (5gb/s per lane), qdr (10gb/s per lane), fdr 10 (1 0.3125gb/s per lane) ethernet: 10gbase-cx4, 10gbase-r, and 1000base-r data rate: 40gb/s fdr10? infiniband 1/10gb/s ? ethernet pci express gen3: serdes @ 8.0gt/s, 8 lanes (2.0 and 1.1 compatible) power and environmental voltage: 12v, 3.3v typ power: passive cables 6.84w active cables 8.14w b b. these power numbers were measured using mellanox cable mc22062xx-0xx-x. max power: passive cables 7.82w active cables 9.32w temperature: operational 0c to 55c non-operational 0c to 70c humidity: 90% relative humidity c c. for both operational a nd non-operational states air flow: 200lfm d d. air flow is measured ~1? from the heat sink between the heat sink and the cooling air inlet. regulatory emc: refer to the following link: www.mellanox.com/related-docs/user_manuals/regulator y_and_compliance_guide.pdf safety: iec/en 60950-1:2006 etsi en 300 019-2-2 iec 60068-2- 64, 29, 32 rohs: rohs-r6 cable support please refer to "mellanox produc ts approved cable lists" at: www.mellanox.com/related -docs/user_manuals/m ellanox_approved_cables.pdf www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 43 a.5 mcx353a-fcbt specifications table 15 - mcx353a-fcbt specifications table physical size: 2.12in. x5.6 in. (53.59mm x 142.25 mm) connector: qsfp+ infiniband (copper and optical) protocol support infiniband: ibta v1.2.1 auto-negotiation a a. the connectx-3 adapters supplement the ibta auto-n egotiation specification to get better bit error rates and longer cable reaches. this s upplemental feature only initiates when connected to another mellanox infiniband product. : 1x/2x/4x sdr (2.5gb/s per lane ), ddr (5gb/s per lane), qdr (10gb/s per lane), fdr10 (10.3125gb/s pe r lane), fdr (1 4.0625gb/s per lane) port ethernet: 10gbase-cx4, 10gbase-r, and 1000base-r, 40gbase-r4 data rate: up to 56gb/s fdr? infiniband 1/10/40gb/s ? ethernet pci express gen3: serdes @ 8.0gt/s, 8 lanes (2.0 and 1.1 compatible) power and environmental voltage: 12v, 3.3v typ power: passive cables 6.71w active cables 8.41w b b. these power numbers were measured using mellanox cable mc2207310-0xx-x. max power: passive cables 8.13w active cables 10.13w temperature: operational 0c to 55c non-operational 0c to 70c humidity: 90% relative humidity c c. for both operational a nd non-operational states air flow: 200lfm d d. air flow is measured ~1? from the heat sink between the heat sink and the cooling air inlet. regulatory emc: refer to the following link: www.mellanox.com/related-docs/user_manuals/regulator y_and_compliance_guide.pdf safety: iec/en 60950-1:2006 etsi en 300 019-2-2 iec 60068-2- 64, 29, 32 rohs: rohs-r6 cable support please refer to "mellanox produc ts approved cable lists" at: www.mellanox.com/related -docs/user_manuals/m ellanox_approved_cables.pdf www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 44 a.6 mcx354a-fcbt specifications table 16 - mcx354a-fcbt specifications table physical size: 2.71in. x5.6 in. (68.90mm x 142.25 mm) connector: qsfp+ infiniband (copper and optical) protocol support infiniband: ibta v1.2.1 auto-negotiation a a. the connectx-3 adapters supplement the ibta auto-n egotiation specification to get better bit error rates and longer cable reaches. this s upplemental feature only initiates when connected to another mellanox infiniband product. : 1x/2x/4x sdr (2.5gb/s per lane ), ddr (5gb/s per lane), qdr (10gb/s per lane), fdr10 (10.3125gb/s pe r lane), fdr (1 4.0625gb/s per lane) port ethernet: 10gbase-cx4, 10gbase-r, and 1000base-r, 40gbase-r4 data rate: up to 56gb/s fdr? infiniband 1/10/40gb/s ? ethernet pci express gen3: serdes @ 8.0gt/s, 8 lanes (2.0 and 1.1 compatible) power and environmental voltage: 12v, 3.3v typ power: passive cables 7.94w active cables 11.34w b b. these power numbers were measured using mellanox cable mc2207310-0xx-x. max power: passive cables 9.35w active cables 11.35w temperature: operational 0c to 55c non-operational 0c to 70c humidity: 90% relative humidity c c. for both operational a nd non-operational states air flow: 200lfm d d. air flow is measured ~1? from the heat sink between the heat sink and the cooling air inlet. regulatory emc: refer to the following link: www.mellanox.com/related-docs/user_manuals/regulator y_and_compliance_guide.pdf safety: iec/en 60950-1:2006 etsi en 300 019-2-2 iec 60068-2- 64, 29, 32 rohs: rohs-r6 cable support please refer to "mellanox produc ts approved cable lists" at: www.mellanox.com/related -docs/user_manuals/m ellanox_approved_cables.pdf www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 45 a.7 board mechanical drawing and dimensions all dimensions are in millimeters. all the mechanical tolerances are +/- 0.1mm figure 10: mechanical drawing of the single-port mcx353a-[qft]cbt 142.24 53.59 40.5 49.09 45.95 12.75 22.61 j1 - i 2 c connector flash jumper adapter card www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 46 figure 11: mechanical drawing of the dual-port mcx354a-[qft]cbt adapter card 142.25 56.15 68.90 45.96 12.75 50.00 64.4 j1 - i 2 c connector flash jumper a.8 regulatory statements for regulatory statements for all connectx?-3 cards please refer to: http://www.mellanox.com/related-docs/user_man uals/reg ulatory_and_compliance_guide.pdf www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 47 appendix b: interface connectors pinout b.1 i 2 c-compatible connector pinout figure 12: compatible connector plug and pinout table 17 - i2c-compatible connector pinout connector pin number signal name 1spsda 2 spscl 3 gnd 4nc 5nc b.2 pci express x8 connector pinout the adapter cards use a standard pci express x8 ed ge connector and the pci express x8 standard pinout according to the pci express 3.0 specification. www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 48 figure 13: pcie connector pinout www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 49 b.3 qsfp+ connector pinout figure 14: connector and cage views www.datasheet.net/ datasheet pdf - http://www..co.kr/
table 18 - connector pin number and name to signal name map connector pin number connector pin name port a signal name 1 gnd gnd 2 txn_2 tx2n 3 txp_2 tx2p 4 gnd gnd 5 txn_4 tx4n 6 txp_4 tx4p 7 gnd gnd 8 modsell_port0 modsell 9 resetl_port0 resetl 10 vccrx 11 scl scl 12 sda sda 13 gnd gnd 14 rxp_3 rx3p 15 rxn_3 rx3n 16 gnd gnd 17 rxp_1 rx1p 18 rxn_1 rx1n 19 gnd gnd 20 gnd gnd 21 rxn_2 rx2n 22 rxp_2 rx2p 23 gnd gnd 24 rxn_4 rx4n 25 rxp_4 rx4p 26 gnd gnd 27 modprsl_port0 mod prsl 28 intl intl 29 vcctx 30 vcc1 31 lpmode_port0 lpmode 32 gnd gnd 33 txp_3 tx3p 34 txn_3 tx3n 35 gnd gnd 36 txp_1 tx1p 37 txn_1 tx1n 38 gnd gnd rev 1.6 mellanox technologies 50 www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 51 appendix c: replacing a tall bracket with a short bracket this section provides instructions on how to remove the tall bracket of a standard mellanox tech - nologies adapter card and replace it with a short one. c.1 replacing a bracket to replace the bracket you will need the following parts: ? the new bracket of the proper height ? one new square gasket for each of the ports ? the 2 screws saved from the removal of the bracket ? the 2 fiber washers saved from the removal of the bracket c.2 removing the existing bracket from the adapter card figure 15: bracket screws screws 1. remove the two screws holding the bracket in place. 2. the bracket comes loose from the card. be careful not to put stress on the leds. 3. save the two screws and the two fiber washers. www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 52 c.3 installing the new bracket 1. remove the paper to expose the adhesive on the gasket. 2. place the gasket for each port onto the new bracke t. ma ke sure to corr ectly align the gasket with the hole in the bracket. 3. if the old gaskets are still on th e card, remove them before installing the new bracket. make sure th at only one gasket per port is used. figure 16: gasket gasket location on adapter card figure 17: gasket installation 1. place the bracket onto the card until the screw holes line up. do not force the bracket onto the card. you may have to gently push the leds using a small screwdriver to align the leds with the holes in the bracket. 2. screw on the bracket using the screws and washers saved from the procedure above step 1. www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 53 figure 18: placing the bracket on the card 3. make sure that the leds are a ligned onto the bracket holes. 4. use a torque driver to apply up to 2 lbs-in torque on the screws. www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 54 appendix d: avertissements de scurit d?installation (warnings in french) 1. instructions d?installation lisez toutes les instructions d?installation avant de brancher le matriel la source d?alimentation lectrique. 2. temprature excessive ce matriel ne doit pas fonctionner dans une zone avec une temprature ambiante dpassant le maximum recommand de 55c (131f). un flux d?air de 200lfm cette temprature ambiante ma ximale est ncessaire. en outre, pour garantir un bon coulement de l?air, laissez au moins 8 cm (3 pouces) d?espace libre autour des ouver - tures de ventilation. 3. orages ? dangers lectriques pendant un orage, il ne faut pas utiliser le matriel et il ne faut pas brancher ou dbrancher les cables. 4. branchement/dbranchemen t des cables en cuivr e les cables en cuivre sont lourds et ne sont pa s flexibles, il faut donc faire trs attention en les branchant et en les dbranchant de s connecteurs. consultez le fabricant des cables pour conna?tre les mises en ga rde et les instructions spciales. 5. installation du matriel ce matriel ne doit tre install, rempl ac ou entretenu que par du personnel form et qualifi. 6. elimination du matriel l?limination de ce matriel doit s?effectuer da ns le respect de toutes les lgislations et rglementations nationales en vigueur. www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 55 7. codes lectriques locaux et nationaux ce matriel doit tre install dans le respect des codes lectriques locaux et nationaux. 8. exposition au rayonnement grave mise en garde ? l'utilisation de commandes ou de rgl ages ou l'excution de procdures autres que ce qui est spcifi dans les prsentes peut engendrer une exposition au rayonnement grave. produit laser de classe 1 ? et rfrences aux normes laser les plus rcentes cei 60 825-1:1993 + a1:1997 + a2:2001 et ne 60825- 1:1994+a1:1996+ a2:2001 www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 56 appendix e: sicherheitshinweise (warnings in german) 1. installationsanleitungen lesen sie alle installationsanleitungen, bevor sie das ger?t an die stromversorgung anschlie?en. 2. bertemperatur dieses ger?t sollte nicht in einem bereich mit einer umgebungstemperatur ber der maximal empfohlenen temperatur von 55c (131f) betrieben werden. es ist ein luft - strom von 200 lfm bei maximaler umgebu ngstempera tur erforderlich. au?erdem sollten mindestens 8 cm (3 in.) freiraum um die belftungs?ffnungen sein, um einen einwandfreien luftstrom zu gew?hrleisten. 3. bei gewitter - elektrische gefahr arbeiten sie w?hrend eines ge witters und blitzschlag nicht am ger?t, schlie?en sie keine kabel an oder ab. 4. anschlie?en/trennen von -kupferkabel kupferkabel sind schwer und nicht flexible . deshalb mssen sie vorsichtig an die anschlsse angebracht bzw. davon getrennt werden. lesen sie die speziellen warnun - gen und anleitungen des kabelherstellers. 5. ger?teinstallation diese ger?t sollte nur von geschultem und qua lifiziertem personal installiert, ausge - tauscht oder gewartet werden. 6. ger?teentsorgung die entsorgung dieses ger?ts sollte unte r beachtung aller nationalen gesetze bestim - mungen erfolgen. 7. regionale und nationale elektrische bestimmungen dieses ger?t sollte unter beachtung der regionalen und nationalen elektrischen bestimmungen installiert werden. t www.datasheet.net/ datasheet pdf - http://www..co.kr/
this equipment should be installed in compliance with local and national elec - trical codes. connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 57 8. strahlenkontak achtung ? nutzung von steuerungen oder einstellungen oder ausfhrung von prozeduren, die hier nicht spezifiziert sind, kann zu gef?hrlichem strahlenkon - takt fhren.. klasse 1 laserprodukt und referenzen zu den aktuellsten lasterstandards : www.datasheet.net/ datasheet pdf - http://www..co.kr/
rev 1.6 mellanox technologies 58 appendix f: advertencias de seguridad para la instalacin (warnings in spanish) 1. instrucciones de instalacin antes de conectar el equipo a la fuente de alimentacin, leer toda s las instrucciones de instalacin. 2. sobrecalentamiento no se debe utilizar el equipo en un rea con una temperatura ambiente superior a la mxima recomendada: 55c(131f). adems, pa ra garantizar una circulacin de aire adecuada, se debe dejar como mnimo un espa cio de 8 cm (3 pulgadas) alrededor de las aberturas de ventilacin. 3. cuando hay rayos: peligro de desc arga elctrica no utilizar el equipo ni cone ctar o desconectar cables dura nte perodos de actividad de rayos. 4. conexin y desconexin del cable copper dado que los cables de cobre son pesados y no son flexibles, su conexin a los conec - tores y su desconexin se deben efectuar con mucho cuidado. para ver advertencias o instrucciones especiales, c onsultar al fabricante del cable. 5. instalacin de equipos la instalacin, el reemplazo y el mantenimiento de este equipo estarn a cargo nica - mente de personal capacitado y competente. 6. eliminacin de equipos la eliminacin definitiva de este equipo se debe efectuar conforme a todas las leyes y reglamentaciones nacionales. www.datasheet.net/ datasheet pdf - http://www..co.kr/
connectx?-3 vpi single and dual qsfp+ port adapter card user manual rev 1.6 mellanox technologies 59 7. cdigos elctricos locales y nacionales este equipo se debe instalar conforme a los cdigos elctricos locales y nacionales. 8. exposicin a niveles de radiacin peligrosos precaucin: el uso de controles o ajustes o la realizacin de procedimientos distintos de los que aqu se especifican podran causar exposicin a niveles de radiacin peligrosos. producto lser de clase 1 y referencia a las normas de lser ms recientes: iec 60825-1:2007/03 y en 60825-1:2007 www.datasheet.net/ datasheet pdf - http://www..co.kr/

▲Up To Search▲

Price & Availability of MT27508A0-FCCR-FV

	To Download MT27508A0-FCCR-FV Datasheet File
If you can't view the Datasheet, Please click here to try to view without PDF Reader .